The Expectation-Maximization and Alternating Minimization Algorithms
نویسنده
چکیده
The Expectation-Maximization (EM) algorithm is a hill-climbing approach to finding a local maximum of a likelihood function [7, 8]. The EM algorithm alternates between finding a greatest lower bound to the likelihood function (the “E Step”), and then maximizing this bound (the “M Step”). The EM algorithm belongs to a broader class of alternating minimization algorithms [6], which includes the Arimoto-Blahut algorithm for calculating channel capacity and rate distortion functions [1, 3], and Cover’s portfolio algorithm to maximize expected log-investment [4].
منابع مشابه
Alternating Minimization as Sequential Unconstrained Minimization: A Survey
Sequential unconstrained minimization is a general iterative method for minimizing a function over a given set. At each step of the iteration we minimize the sum of the objective function and an auxiliary function. The aim is to select the auxiliary functions so that, at least, we get convergence in function value to the constrained minimum. The SUMMA is a broad class of these methods for which...
متن کاملAn Hybrid Interior Point Algorithm for Performing Alternating Minimizations an Hybrid Interior Point Algorithm for Performing Alternating Minimizations
Optimization using alternating minimization (AM) alternately minimizes an objective function with respect to two disjoint subsets of a function's parameters. AM algorithms are useful for optimization problems which can be decomposed into convex subproblems. AM algorithms have recently appeared in the neural computing community. This paper examines the use of recently developed Interior Point (I...
متن کاملMirror-Descent-like Algorithms for Submodular Optimization
In this paper we develop a framework of submodular optimization algorithms in line with the mirror-descent style of algorithms for convex optimization. We use the fact that a submodular function has both a subdifferential and a superdifferential, which enables us to formulate algorithms for both submodular minimization and maximization. This reveals a unifying framework for a number of submodul...
متن کاملGeneral Convergent Expectation Maximization (em)-type Algorithms for Image Reconstruction
Obtaining high quality images is very important in many areas of applied sciences, such as medical imaging, optical microscopy, and astronomy. Image reconstruction can be considered as solving the ill-posed and inverse problem y = Ax+n, where x is the image to be reconstructed and n is the unknown noise. In this paper, we propose general robust expectation maximization (EM)-Type algorithms for ...
متن کاملAlternating minimization and Boltzmann machine learning
Training a Boltzmann machine with hidden units is appropriately treated in information geometry using the information divergence and the technique of alternating minimization. The resulting algorithm is shown to be closely related to gradient descent Boltzmann machine learning rules, and the close relationship of both to the EM algorithm is described. An iterative proportional fitting procedure...
متن کامل